Benchmarking reports

from sklearn_benchmarks.report import Reporting
import pandas as pd
pd.set_option('display.max_colwidth', None)
pd.set_option('display.max_columns', None)
pd.set_option('display.max_rows', None)

reporting = Reporting(config_file_path="config.yml")
reporting.run()

Time report

	hour	min	sec
algo
KNeighborsClassifier	0.0	13.0	33.672632
daal4py_KNeighborsClassifier	0.0	5.0	11.022916
KNeighborsClassifier_kd_tree	0.0	6.0	0.327172
daal4py_KNeighborsClassifier_kd_tree	0.0	1.0	46.091624
KMeans_tall	0.0	1.0	45.454342
daal4py_KMeans_tall	0.0	1.0	19.288143
KMeans_short	0.0	0.0	23.942625
daal4py_KMeans_short	0.0	0.0	13.709325
LogisticRegression	0.0	1.0	8.222998
daal4py_LogisticRegression	0.0	1.0	0.455939
Ridge	0.0	0.0	26.110131
daal4py_Ridge	0.0	0.0	8.732427
total	0.0	32.0	57.122206

KNeighborsClassifier vs daal4py

All estimators share the following hyperparameters:

	value
algorithm	brute

	estimator	function	mean_sklearn	stdev_sklearn	n_samples_train	n_samples	n_features	n_jobs	n_neighbors	throughput	latency	accuracy_score_sklearn	accuracy_score_daal4py	mean_daal4py	stdev_daal4py	speedup	stdev_speedup	profiling
0	KNeighborsClassifier	fit	0.132	0.004	1000000	1000000	100	-1	1	6.073	NaN	0.987	0.981	0.484	0.018	0.272	0.013	See
1	KNeighborsClassifier	predict	30.689	0.000	1000000	1000	100	-1	1	0.000	0.031	0.987	0.981	3.787	0.015	8.104	0.032	See
2	KNeighborsClassifier	predict	0.191	0.016	1000000	1	100	-1	1	0.004	0.000	0.987	0.981	0.100	0.002	1.913	0.167	See
3	KNeighborsClassifier	fit	0.132	0.002	1000000	1000000	100	-1	5	6.060	NaN	0.987	0.981	0.490	0.011	0.269	0.007	See
4	KNeighborsClassifier	predict	38.225	0.000	1000000	1000	100	-1	5	0.000	0.038	0.987	0.981	3.788	0.011	10.090	0.028	See
5	KNeighborsClassifier	predict	0.199	0.013	1000000	1	100	-1	5	0.004	0.000	0.987	0.981	0.102	0.004	1.943	0.150	See
6	KNeighborsClassifier	fit	0.129	0.003	1000000	1000000	100	-1	100	6.197	NaN	0.987	0.981	0.485	0.013	0.266	0.009	See
7	KNeighborsClassifier	predict	38.085	0.000	1000000	1000	100	-1	100	0.000	0.038	0.987	0.981	3.852	0.016	9.888	0.040	See
8	KNeighborsClassifier	predict	0.199	0.011	1000000	1	100	-1	100	0.004	0.000	0.987	0.981	0.100	0.002	1.985	0.121	See
9	KNeighborsClassifier	fit	0.140	0.002	1000000	1000000	100	1	1	5.726	NaN	0.987	0.981	0.480	0.012	0.291	0.009	See
10	KNeighborsClassifier	predict	15.145	0.162	1000000	1000	100	1	1	0.000	0.015	0.987	0.981	3.787	0.021	3.999	0.048	See
11	KNeighborsClassifier	predict	0.192	0.006	1000000	1	100	1	1	0.004	0.000	0.987	0.981	0.101	0.003	1.900	0.077	See
12	KNeighborsClassifier	fit	0.123	0.002	1000000	1000000	100	1	5	6.529	NaN	0.987	0.981	0.483	0.017	0.254	0.010	See
13	KNeighborsClassifier	predict	24.903	0.146	1000000	1000	100	1	5	0.000	0.025	0.987	0.981	3.784	0.010	6.581	0.042	See
14	KNeighborsClassifier	predict	0.203	0.008	1000000	1	100	1	5	0.004	0.000	0.987	0.981	0.101	0.003	2.018	0.093	See
15	KNeighborsClassifier	fit	0.130	0.002	1000000	1000000	100	1	100	6.148	NaN	0.987	0.981	0.500	0.010	0.260	0.006	See
16	KNeighborsClassifier	predict	24.747	0.062	1000000	1000	100	1	100	0.000	0.025	0.987	0.981	3.849	0.023	6.429	0.042	See
17	KNeighborsClassifier	predict	0.216	0.012	1000000	1	100	1	100	0.004	0.000	0.987	0.981	0.099	0.002	2.185	0.131	See
18	KNeighborsClassifier	fit	0.057	0.000	1000000	1000000	2	-1	1	0.281	NaN	0.987	0.981	0.103	0.003	0.555	0.017	See
19	KNeighborsClassifier	predict	23.739	0.066	1000000	1000	2	-1	1	0.000	0.024	0.987	0.981	0.799	0.008	29.700	0.326	See
20	KNeighborsClassifier	predict	0.020	0.001	1000000	1	2	-1	1	0.001	0.000	0.987	0.981	0.004	0.000	4.738	0.518	See
21	KNeighborsClassifier	fit	0.057	0.001	1000000	1000000	2	-1	5	0.280	NaN	0.987	0.981	0.103	0.005	0.553	0.030	See
22	KNeighborsClassifier	predict	31.803	0.000	1000000	1000	2	-1	5	0.000	0.032	0.987	0.981	0.810	0.023	39.241	1.098	See
23	KNeighborsClassifier	predict	0.029	0.003	1000000	1	2	-1	5	0.001	0.000	0.987	0.981	0.004	0.001	6.624	1.334	See
24	KNeighborsClassifier	fit	0.057	0.001	1000000	1000000	2	-1	100	0.281	NaN	0.987	0.981	0.103	0.002	0.551	0.011	See
25	KNeighborsClassifier	predict	31.374	0.000	1000000	1000	2	-1	100	0.000	0.031	0.987	0.981	0.868	0.009	36.143	0.370	See
26	KNeighborsClassifier	predict	0.028	0.001	1000000	1	2	-1	100	0.001	0.000	0.987	0.981	0.004	0.000	6.349	0.622	See
27	KNeighborsClassifier	fit	0.058	0.003	1000000	1000000	2	1	1	0.275	NaN	0.987	0.981	0.104	0.003	0.560	0.030	See
28	KNeighborsClassifier	predict	10.521	0.039	1000000	1000	2	1	1	0.000	0.011	0.987	0.981	0.797	0.009	13.201	0.161	See
29	KNeighborsClassifier	predict	0.016	0.001	1000000	1	2	1	1	0.001	0.000	0.987	0.981	0.004	0.000	3.685	0.360	See
30	KNeighborsClassifier	fit	0.058	0.003	1000000	1000000	2	1	5	0.278	NaN	0.987	0.981	0.103	0.002	0.556	0.036	See
31	KNeighborsClassifier	predict	18.631	0.014	1000000	1000	2	1	5	0.000	0.019	0.987	0.981	0.802	0.010	23.223	0.285	See
32	KNeighborsClassifier	predict	0.022	0.001	1000000	1	2	1	5	0.001	0.000	0.987	0.981	0.004	0.000	5.305	0.476	See
33	KNeighborsClassifier	fit	0.058	0.001	1000000	1000000	2	1	100	0.278	NaN	0.987	0.981	0.104	0.003	0.556	0.020	See
34	KNeighborsClassifier	predict	18.577	0.020	1000000	1000	2	1	100	0.000	0.019	0.987	0.981	0.865	0.007	21.472	0.169	See
35	KNeighborsClassifier	predict	0.022	0.001	1000000	1	2	1	100	0.001	0.000	0.987	0.981	0.004	0.000	5.168	0.421	See

KNeighborsClassifier_kd_tree vs daal4py

All estimators share the following hyperparameters:

	value
algorithm	kd_tree

	estimator	function	mean_sklearn	stdev_sklearn	n_samples_train	n_samples	n_features	n_jobs	n_neighbors	throughput	latency	accuracy_score_sklearn	accuracy_score_daal4py	mean_daal4py	stdev_daal4py	speedup	stdev_speedup	profiling
0	KNeighborsClassifier_kd_tree	fit	2.821	0.034	1000000	1000000	10	-1	1	0.028	NaN	0.986	0.987	0.741	0.012	3.807	0.079	See
1	KNeighborsClassifier_kd_tree	predict	0.414	0.009	1000000	1000	10	-1	1	0.000	0.000	0.986	0.987	0.116	0.004	3.573	0.146	See
2	KNeighborsClassifier_kd_tree	predict	0.005	0.002	1000000	1	10	-1	1	0.015	0.000	0.986	0.987	0.000	0.000	25.850	14.737	See
3	KNeighborsClassifier_kd_tree	fit	2.839	0.045	1000000	1000000	10	-1	5	0.028	NaN	0.986	0.987	0.745	0.007	3.810	0.070	See
4	KNeighborsClassifier_kd_tree	predict	0.805	0.011	1000000	1000	10	-1	5	0.000	0.001	0.986	0.987	0.212	0.002	3.804	0.062	See
5	KNeighborsClassifier_kd_tree	predict	0.003	0.001	1000000	1	10	-1	5	0.023	0.000	0.986	0.987	0.000	0.000	7.227	3.438	See
6	KNeighborsClassifier_kd_tree	fit	2.857	0.041	1000000	1000000	10	-1	100	0.028	NaN	0.986	0.987	0.700	0.006	4.080	0.068	See
7	KNeighborsClassifier_kd_tree	predict	2.590	0.020	1000000	1000	10	-1	100	0.000	0.003	0.986	0.987	0.646	0.012	4.008	0.080	See
8	KNeighborsClassifier_kd_tree	predict	0.006	0.001	1000000	1	10	-1	100	0.014	0.000	0.986	0.987	0.003	0.001	1.723	0.577	See
9	KNeighborsClassifier_kd_tree	fit	2.878	0.030	1000000	1000000	10	1	1	0.028	NaN	0.986	0.987	0.748	0.014	3.849	0.083	See
10	KNeighborsClassifier_kd_tree	predict	0.754	0.015	1000000	1000	10	1	1	0.000	0.001	0.986	0.987	0.117	0.004	6.432	0.250	See
11	KNeighborsClassifier_kd_tree	predict	0.002	0.001	1000000	1	10	1	1	0.053	0.000	0.986	0.987	0.000	0.000	8.138	5.041	See
12	KNeighborsClassifier_kd_tree	fit	2.847	0.032	1000000	1000000	10	1	5	0.028	NaN	0.986	0.987	0.700	0.008	4.065	0.064	See
13	KNeighborsClassifier_kd_tree	predict	1.428	0.019	1000000	1000	10	1	5	0.000	0.001	0.986	0.987	0.210	0.003	6.807	0.136	See
14	KNeighborsClassifier_kd_tree	predict	0.002	0.000	1000000	1	10	1	5	0.049	0.000	0.986	0.987	0.000	0.000	3.364	1.443	See
15	KNeighborsClassifier_kd_tree	fit	2.835	0.035	1000000	1000000	10	1	100	0.028	NaN	0.986	0.987	0.745	0.006	3.805	0.058	See
16	KNeighborsClassifier_kd_tree	predict	4.690	0.030	1000000	1000	10	1	100	0.000	0.005	0.986	0.987	0.656	0.010	7.147	0.121	See
17	KNeighborsClassifier_kd_tree	predict	0.004	0.001	1000000	1	10	1	100	0.019	0.000	0.986	0.987	0.002	0.001	1.751	0.613	See
18	KNeighborsClassifier_kd_tree	fit	0.772	0.004	1000000	1000000	2	-1	1	0.021	NaN	0.986	0.987	0.479	0.005	1.610	0.019	See
19	KNeighborsClassifier_kd_tree	predict	0.033	0.001	1000000	1000	2	-1	1	0.000	0.000	0.986	0.987	0.001	0.000	36.190	19.780	See
20	KNeighborsClassifier_kd_tree	predict	0.003	0.000	1000000	1	2	-1	1	0.006	0.000	0.986	0.987	0.000	0.000	14.157	6.474	See
21	KNeighborsClassifier_kd_tree	fit	0.772	0.008	1000000	1000000	2	-1	5	0.021	NaN	0.986	0.987	0.483	0.006	1.598	0.027	See
22	KNeighborsClassifier_kd_tree	predict	0.035	0.001	1000000	1000	2	-1	5	0.000	0.000	0.986	0.987	0.001	0.000	29.759	6.300	See
23	KNeighborsClassifier_kd_tree	predict	0.002	0.000	1000000	1	2	-1	5	0.006	0.000	0.986	0.987	0.000	0.000	13.773	6.072	See
24	KNeighborsClassifier_kd_tree	fit	0.778	0.009	1000000	1000000	2	-1	100	0.021	NaN	0.986	0.987	0.478	0.008	1.629	0.032	See
25	KNeighborsClassifier_kd_tree	predict	0.057	0.003	1000000	1000	2	-1	100	0.000	0.000	0.986	0.987	0.008	0.001	7.428	0.719	See
26	KNeighborsClassifier_kd_tree	predict	0.003	0.001	1000000	1	2	-1	100	0.006	0.000	0.986	0.987	0.000	0.000	15.367	8.977	See
27	KNeighborsClassifier_kd_tree	fit	0.776	0.008	1000000	1000000	2	1	1	0.021	NaN	0.986	0.987	0.485	0.007	1.600	0.028	See
28	KNeighborsClassifier_kd_tree	predict	0.032	0.001	1000000	1000	2	1	1	0.001	0.000	0.986	0.987	0.001	0.000	38.693	11.074	See
29	KNeighborsClassifier_kd_tree	predict	0.001	0.000	1000000	1	2	1	1	0.019	0.000	0.986	0.987	0.000	0.000	5.232	2.594	See
30	KNeighborsClassifier_kd_tree	fit	0.775	0.009	1000000	1000000	2	1	5	0.021	NaN	0.986	0.987	0.479	0.005	1.618	0.025	See
31	KNeighborsClassifier_kd_tree	predict	0.034	0.001	1000000	1000	2	1	5	0.000	0.000	0.986	0.987	0.001	0.001	25.703	12.070	See
32	KNeighborsClassifier_kd_tree	predict	0.001	0.000	1000000	1	2	1	5	0.019	0.000	0.986	0.987	0.000	0.000	2.670	1.927	See
33	KNeighborsClassifier_kd_tree	fit	0.775	0.005	1000000	1000000	2	1	100	0.021	NaN	0.986	0.987	0.488	0.016	1.586	0.052	See
34	KNeighborsClassifier_kd_tree	predict	0.058	0.003	1000000	1000	2	1	100	0.000	0.000	0.986	0.987	0.007	0.001	7.990	0.800	See
35	KNeighborsClassifier_kd_tree	predict	0.001	0.000	1000000	1	2	1	100	0.018	0.000	0.986	0.987	0.000	0.000	5.170	2.518	See

KMeans_tall vs daal4py

All estimators share the following hyperparameters:

	value
algorithm	full
n_clusters	3
max_iter	30
n_init	1
tol	0.0

	estimator	function	mean_sklearn	stdev_sklearn	n_samples_train	n_samples	n_features	init	n_iter_sklearn	throughput	latency	adjusted_rand_score_sklearn	n_iter_daal4py	adjusted_rand_score_daal4py	mean_daal4py	stdev_daal4py	speedup	stdev_speedup	profiling
0	KMeans_tall	fit	0.617	0.014	1000000	1000000	2	k-means++	30	0.778	NaN	0.002	30	0.002	0.310	0.010	1.991	0.080	See
1	KMeans_tall	predict	0.002	0.000	1000000	1000	2	k-means++	30	0.011	0.0	0.002	30	0.002	0.000	0.000	7.069	2.799	See
2	KMeans_tall	predict	0.002	0.000	1000000	1	2	k-means++	30	0.010	0.0	0.002	30	0.002	0.000	0.000	9.187	4.130	See
3	KMeans_tall	fit	0.524	0.010	1000000	1000000	2	random	30	0.916	NaN	0.002	30	0.002	0.269	0.005	1.947	0.053	See
4	KMeans_tall	predict	0.002	0.000	1000000	1000	2	random	30	0.010	0.0	0.002	30	0.002	0.000	0.000	7.200	3.142	See
5	KMeans_tall	predict	0.002	0.000	1000000	1	2	random	30	0.010	0.0	0.002	30	0.002	0.000	0.000	5.523	2.430	See
6	KMeans_tall	fit	7.051	0.093	1000000	1000000	100	k-means++	30	3.404	NaN	0.002	30	0.002	3.842	0.036	1.835	0.030	See
7	KMeans_tall	predict	0.002	0.000	1000000	1000	100	k-means++	30	0.463	0.0	0.002	30	0.002	0.000	0.000	5.336	1.996	See
8	KMeans_tall	predict	0.002	0.000	1000000	1	100	k-means++	30	0.518	0.0	0.002	30	0.002	0.000	0.000	8.067	3.154	See
9	KMeans_tall	fit	6.505	0.048	1000000	1000000	100	random	30	3.690	NaN	0.002	30	0.002	3.590	0.046	1.812	0.027	See
10	KMeans_tall	predict	0.002	0.000	1000000	1000	100	random	30	0.452	0.0	0.002	30	0.002	0.000	0.000	5.448	1.883	See
11	KMeans_tall	predict	0.002	0.000	1000000	1	100	random	30	0.529	0.0	0.002	30	0.002	0.000	0.000	7.095	2.970	See

KMeans_short vs daal4py

All estimators share the following hyperparameters:

	value
algorithm	full
n_clusters	300
max_iter	30
n_init	1
tol	0.0

	estimator	function	mean_sklearn	stdev_sklearn	n_samples_train	n_samples	n_features	init	n_iter_sklearn	throughput	latency	adjusted_rand_score_sklearn	n_iter_daal4py	adjusted_rand_score_daal4py	mean_daal4py	stdev_daal4py	speedup	stdev_speedup	profiling
0	KMeans_short	fit	0.318	0.010	10000	10000	2	k-means++	27	0.014	NaN	0.006	25	0.007	0.144	0.007	2.212	0.130	See
1	KMeans_short	predict	0.002	0.001	10000	1000	2	k-means++	27	0.008	0.0	0.006	25	0.007	0.001	0.000	2.956	1.153	See
2	KMeans_short	predict	0.002	0.000	10000	1	2	k-means++	27	0.011	0.0	0.006	25	0.007	0.000	0.000	8.637	4.314	See
3	KMeans_short	fit	0.131	0.001	10000	10000	2	random	30	0.037	NaN	0.006	30	0.007	0.070	0.004	1.876	0.101	See
4	KMeans_short	predict	0.002	0.000	10000	1000	2	random	30	0.008	0.0	0.006	30	0.007	0.001	0.000	3.064	0.524	See
5	KMeans_short	predict	0.002	0.001	10000	1	2	random	30	0.008	0.0	0.006	30	0.007	0.000	0.000	12.365	7.315	See
6	KMeans_short	fit	1.051	0.054	10000	10000	100	k-means++	20	0.152	NaN	0.006	21	0.007	0.613	0.036	1.715	0.135	See
7	KMeans_short	predict	0.004	0.001	10000	1000	100	k-means++	20	0.226	0.0	0.006	21	0.007	0.001	0.000	2.427	0.874	See
8	KMeans_short	predict	0.002	0.000	10000	1	100	k-means++	20	0.499	0.0	0.006	21	0.007	0.000	0.000	6.989	3.897	See
9	KMeans_short	fit	0.327	0.040	10000	10000	100	random	22	0.539	NaN	0.006	19	0.007	0.287	0.053	1.138	0.253	See
10	KMeans_short	predict	0.003	0.000	10000	1000	100	random	22	0.264	0.0	0.006	19	0.007	0.001	0.000	2.082	0.209	See
11	KMeans_short	predict	0.002	0.000	10000	1	100	random	22	0.479	0.0	0.006	19	0.007	0.000	0.000	8.257	3.478	See

LogisticRegression vs daal4py

All estimators share the following hyperparameters:

	value
penalty	l2
dual	False
tol	0.0001
C	1.0
fit_intercept	True
intercept_scaling	1
class_weight	NaN
random_state	NaN
solver	lbfgs
max_iter	100
multi_class	auto
verbose	0
warm_start	False
n_jobs	NaN
l1_ratio	NaN

	estimator	function	mean_sklearn	stdev_sklearn	n_samples_train	n_samples	n_features	class_weight	l1_ratio	n_jobs	random_state	n_iter	throughput	latency	accuracy_score	mean_daal4py	stdev_daal4py	speedup	stdev_speedup	profiling
0	LogisticRegression	fit	15.250	0.035	1000000	1000000	100	NaN	NaN	NaN	NaN	[20]	[-0.07737027]	NaN	0.3	15.182	0.032	1.004	0.003	See
1	LogisticRegression	predict	0.000	0.000	1000000	1000	100	NaN	NaN	NaN	NaN	[20]	2.2430589933545066	0.0	0.3	0.000	0.000	0.873	0.375	See
2	LogisticRegression	predict	0.000	0.000	1000000	1	100	NaN	NaN	NaN	NaN	[20]	8.208596861893794	0.0	0.3	0.000	0.000	0.383	0.312	See
3	LogisticRegression	fit	1.135	0.027	1000	1000	10000	NaN	NaN	NaN	NaN	[26]	[1.83291975]	NaN	0.3	1.137	0.046	0.998	0.046	See
4	LogisticRegression	predict	0.002	0.000	1000	100	10000	NaN	NaN	NaN	NaN	[26]	3.4440048507013428	0.0	0.3	0.004	0.000	0.531	0.089	See
5	LogisticRegression	predict	0.000	0.000	1000	1	10000	NaN	NaN	NaN	NaN	[26]	42.88831345827943	0.0	0.3	0.001	0.000	0.226	0.300	See

Ridge vs daal4py

All estimators share the following hyperparameters:

	value
alpha	1.0
fit_intercept	True
normalize	deprecated
copy_X	True
max_iter	NaN
tol	0.001
solver	auto
random_state	NaN

	estimator	function	mean_sklearn	stdev_sklearn	n_samples_train	n_samples	n_features	max_iter	random_state	throughput	latency	r2_score	mean_daal4py	stdev_daal4py	speedup	stdev_speedup	profiling
0	Ridge	fit	0.278	0.005	1000	1000	10000	NaN	NaN	0.288	NaN	1.0	0.285	0.002	0.976	0.020	See
1	Ridge	predict	0.011	0.000	1000	1000	10000	NaN	NaN	7.206	0.0	1.0	0.018	0.001	0.629	0.037	See
2	Ridge	predict	0.000	0.000	1000	1	10000	NaN	NaN	708.529	0.0	1.0	0.000	0.000	0.724	0.503	See
3	Ridge	fit	1.177	0.044	1000000	1000000	100	NaN	NaN	0.680	NaN	1.0	0.328	0.002	3.586	0.136	See
4	Ridge	predict	0.000	0.000	1000000	1000	100	NaN	NaN	4.104	0.0	1.0	0.000	0.000	0.801	0.440	See
5	Ridge	predict	0.000	0.000	1000000	1	100	NaN	NaN	8.687	0.0	1.0	0.000	0.000	0.642	0.551	See

Environment information

{
  "system_info": {
    "python": "3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27)  [GCC 9.3.0]",
    "executable": "/usr/share/miniconda/envs/sklbench/bin/python",
    "machine": "Linux-5.4.0-1046-azure-x86_64-with-glibc2.10"
  },
  "dependencies_info": {
    "pip": "21.1.1",
    "setuptools": "49.6.0.post20210108",
    "sklearn": "1.0.dev0",
    "numpy": "1.20.2",
    "scipy": "1.6.3",
    "Cython": null,
    "pandas": "1.2.4",
    "matplotlib": null,
    "joblib": "1.0.1",
    "threadpoolctl": "2.1.0"
  },
  "threadpool_info": [
    {
      "filepath": "/usr/share/miniconda/envs/sklbench/lib/libopenblasp-r0.3.15.so",
      "prefix": "libopenblas",
      "user_api": "blas",
      "internal_api": "openblas",
      "version": "0.3.15",
      "num_threads": 2,
      "threading_layer": "pthreads"
    },
    {
      "filepath": "/usr/share/miniconda/envs/sklbench/lib/python3.8/site-packages/scikit_learn.libs/libgomp-f7e03b3e.so.1.0.0",
      "prefix": "libgomp",
      "user_api": "openmp",
      "internal_api": "openmp",
      "version": null,
      "num_threads": 2
    }
  ],
  "cpu_count": 2
}